Instantiating Thematic Roles with a Recurrent Neural Network

NOTE: This report is available both on paper and in electronic form. The electronic version may lack illustrations and special characters. The paper version is available from The University of Georgia. Human languages have both a surface structure and a deep structure. The surface structure of a language can be described by grammatical rules which generate its well-formed sentences. Deep structure can be described by a set of thematic roles which specify the meaning of a sentence independent of any particular language. In this paper, I show that a recurrent neural network can learn to map English sentences generated by a unification-based grammar onto their appropriate thematic roles. Two models are evaluated: (1) a simple recurrent network like those of Jordan (1986) and Elman (1990), and (2) a modified recurrent network which incorporates a time-varying training signal, direct connections between its input and output units, and two separate state layers. It is found that the modified network learns faster and more completely than the simple recurrent network.